Apr 24 2006 9:19PM Kendyl Roman 



4087398882 



p . 35 



1 

SPE C IFI C ATION WITH MARKUPS SMQWiNO CHANGES 



Patent Application of 
Paul Raposo 

TIT LEi BENCHMARKIN G SURVEYS 
TITLE: DATA-DRIVEN INTERNET SURVEY SYSTEM 

BACKGROUND - FIELD OF THE INVENTION 

This invention relates to benchmarking different surveys, specifically benchmarking 

of Internet based surveys. 

BACKGROUND-SURVEYS 

Surveys or polls are commonly used to collect information. Over time a survey may 

be modified. Also once a survey is complete another different survey may be used to collect 
follow-up information. It is difficult to compare different surveys. It would be valuable to 
be able to simply see the results of a survey as a number or an index that would have 
meaning over time. 

SUMMARY OF THE INVENTION 

In accordance with the present invention a method of creating benchmarks for 

surveys that are used to graphically display over time and over multiple different surveys an 
overall trend. 

Objects and Advantages 

Accordingly, beside the objects and advantages of the method described in my patent 

above, some additional objects and advantages of the present invention are: 
1. to provide a method of assigning goals and weights to survey elements. 
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2. to provide a method of determining an index based on survey results against said 
goals and weights. 

3. to provide a graphical view of the index of related surveys over time to show trends. 

4. to provide a professional web-based survey system that supports various types of 
survey questions and also supports the necessary goals and weights necessary to 
produce an index for each survey. 

DRAWING FIGURES 

In the drawings, closely related figures have the same number but different 

alphabetic suffixes. 

Fig 1 illustrates the survey system process flow including the provisions for 
benchmarking. 

F i g 2 io a otructurnl vi e w of the survey ayst e m of Fig 1 

F i gs 3A through 3B arc three versiona - of - a - Hrex - ample survey, 

Fig A is a tabl e illustrating th e goals, w e ights, and index calculat io ns for the 
quooriono in the example our - vey of jng 3A, 

Fig 5 is u graph oho wing th e resulto and ind e x for the example surv e y of Fig 4. 

F i g 6 i s a graph showing the mdex trend of the t h ree vers i ons of th e exampl e ourv e y 
of Figfi 3A through 3C. 

Reference Numerals In Drawings 

100 secure server 
1 10 server web site 
120 respondent member 
130 client user 
140 panelist 
150 client web site 
160 survey target 
170 network 
180 survey builder 

209 databa se 

230 re s pondent comput e r 

230 cli e nt computer 

240 pa nelist computer 

260 survey target comput e r 

302 surv e y 2 



303 s urvey 3 

340 popular i ty obj e ct i ve 

344 popularity qu e stio n- 

344 o in c e re 

345 charismatic 

324 p e rformance qu e stion 

323 economy 

324 dom e stic policy 



326 health care 

£3G re el e ction objecti ve 

434 re-^te et question 
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good apeulcer 



socio] security iosue 
def e nce 



3S£- 



ri m nn.litnr 

the war 



360 stand obj c otiv e 

344 stand question 

343 seeia l security inv e stment 

364 oocial program * ; 

345 modified social security 

366 estate tax 




449 goal answer vo + uooo i umft 

430a - re s p o nd e nt 1 respon s e 



430b — r e spo n dent 2 respons e 



4304 — respondent A rcoponoc 
440a — r esp ond en t I calculation 
440b — respondent 3 oulouiat i oTv 
440e — respond e nt 3 calculation 
44©d — r espondent 4 calculation 
4S0 mdex row 

J j>T_»l I v« VJriT » i tiviv 

respondent 2 i ndex 
4S0e — respondent 3 -i ndex 
4504 — respondent 4 i ndex 

460 overall surv e y ind e x. 

460a — survey 1 index 
460b — su rvey 2 ind e x 
460e — surv e y 3 index 

§00 survey I graph 

£4-0 3urvey benchmurk 

530 results l i n e 

600 survey trend graph 

610 benchmark l i n e 

620 trend lin e 
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DESCRIPTION OF THE INVENTION 

Surveys are all different, even different versions of the same survey. These 

differences can be due to questions added, questions removed or questions modified. 
Question modifications can include changing wording, changing options or changing types 
(changing a close ended question to an open ended question and vice versa). 

Benchmarks provide a quantifiable way to conceptually grade and measure the 
success of a survey against goals. 

One- way to undorotnud the oonoopt of n purvey benchmark i ff to con s ider ah example 
of a riv e r which floods from tim e to time, A. surveyor of a town through which the river 
pass e s - pl aces a mark on o permanent obj e ct n e xt to tho river (e.g. a bridg e pies, - or th e 
"bench" of the river bank). Th e benchmark serves as a r sf o rence for wat e r lev e l 
observations. The b e nchmark indicates when the river i u at 100% of its capacity. Th e 
b e nchmark often has a scale to measure the number of foot below flood level and th e 
numborof feet nbove flood lev e l. When h e avy rains fall, the r e sidents of the town can 
monitor th e ris e of the river against th e benchmark and mak e d e cisions based on th e tr e nd - . 
If the river is below the 100% mark and the trend i G d e creasing, the reoid e ntsoomfortably 
wi ll stay home.— However, if the river w rioing close to the 100% mark and i t i s still raining, 
the-fesi dento may evacuate low l y i ng ar e as. When th e river ris e s abov e the benchmark, th e 
resid e nts can predict die e xtent of th e flooding by the measur e of the water level abov e the 
benchmark. Wh i le there ore t i mes wh e n observat i on of ths benchmark and calcu l ating i tc 
trend ar e not urgent, th e r e ar e oth e r tim e s whan i mmediat e results and trend analysis i s 
cr i tical to decision making. 

In th e e xample of the - rivor benchmark, the same thing, the water level, ic being 
measured ov e r time. However, as summarized above, in a question bnsod surv e y ^-th^ 
questions change over time. Th e present inv e nt i on prov i d e s s ystems and methoda for 
apply i ng the concept of benchmarks to question based surveys. 

.These benchmarks transcend surveys and differences within these surveys including 
question changes and wording. The benchmark uses an index that works in much the same 
way that a stock market index works (by selecting key stocks, weighting them and 
monitoring their activity as an indication of the overall stock market activity). However 
unlik e the river e xampl e which measures only one thing (i.e. th e wat e r level), and unlike the 
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stock index example wh i ch ua e s an e xist i ng number for the sam e thing (i.e. the price of a 
c e rtain stock) i n the ind e x ua i oulfttion, the present invention provide s a surv e y s ystem which 
i ncludes a m e ans for assigning goa l answ e r win es attd-weigfot s- to - d i fPe r- m g survey questions 
to produce ai> i ndex which transoendo tho diff e rences in th e questions ov e r time. 

It is not uncommon for surveys to be conducted more than once* Not only can 
benchmarking help measure the objectives of a survey and compare surveys, but also, by 
tracking them over time, measure progress towards overall market research objectives. 

Fig 1-Survey System Overview 

The survey system provides a computer network that can be used to design surveys 

in a way that the survey results can be tallied to determine an index, and the index can be 

correlated with the indices of other related surveys to produce a graph of the trends of the 

success toward the goals of the surveys. In the preferred embodiment, the survey system is 

implemented as a web-based survey system. Fig 1 illustrates the process flow of the survey 

system. Fig. 1 includes information on the physical structure of the computer network of the 

preferred embodiment, the software and database structure, the functions that the various 

components and users may use, and the data flow paths for the various actions. In the 

preferred embodiment, the survey system comprises a secure server 100, a server web site 

1 10, a client web site 150, each connected to a network 170- After client users 130 sign up 

for membership, they are able to create and modify surveys by securely accessing a survey 

builder 180. Surveys are defined using predetermined formats for various types of 

questions. Client users have objectives for their surveys. Each question is assigned a goal 

answer value relative to one of the objectives. The survey system provides a means for 

assigning weights that are used to tally the responses to determine an index for each version 

of a survey. The survey system provides reports showing the results of the survey, including 

graphs of the indices of the various versions of surveys showing the trend of the success 

towards the goals and objectives. 

F i g i wil l be diooufised more in detail below. 

Fig 3 Survey System Computer Network Structure 

Fig 2 illuGtrate s- d> e structural view of a preferred embodiment of the ourvey system 

of F i g 1, Tha survey system compris e s a s e cure server 100, a s e rver w e b c i te 1 10 g a client 
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w e b sir e 1 50, each connect e d to a network 170. The secur e server 100 inc l udes a databaoo 
20 0. ht the pref e rred e mbodim e nt, the database 200 is an Orucie database management 
system that includes a member database, a c l i e nt survey - database, surv e y t e mplat e s 
databas e , n cl i e n t raopond e nt/other dnto database, and a reopondont member database. 

Th e survey builder 1 8 0 is access e d via di e- scetiro aorvor web site 1 - I0.- The purvey 
buildef 480 securely accesses th e databas e 200 to maintain - data regard i ng the members - , - 
client surv e y s , survey temp l ates, cli e nt respondent: demographies, and respondents. The 
s eeure s erver 100 <n t d - server-w e b site HQ could be impl e m e nt e d on a s ing l e computer or 
multiple computers. If implemented on separate computers (as shown i n Ftg 2), th e s e cur e 
conn e ction coukl b e impl e mented as a dir e ct ^ conn e ction, a priva t e - ne tw ork, or a virtua l 
pr i vat e n e twork (VPN), as known in th e art. In some embodim e nts, th e secur e serv e r - 400 
hosts the client web site 150, 

Th e client us e r 130 can n l oo speoify demographics for th e types of i ndividua l s thay 
want to respond to the survey. After a survey has boon orontcd, or modif i ed to cr e ate 
another version of an existing survey, th e el i e at user J 30 can post th e surv e y to the cli e nt' s 
web site 150, The responses to survey quontionn are returned to the coour e s e rver 100 and 
database 200 via network «1 70. 

Fig 2 ohowa different type of individuals who may respond to surveys. Respond e nt 
members 120. cli e nt users 130, pan e lists 1 40, and other s urvey targets 160 may respond to 
surv e ys via their own computers, name l y, respondent computors !I ^O t - cli e nt user computers 
£30, pan e list computers 3d0v and surv e y targ e t computers 360, roapooti voly> The respondent 
computers 220, client user computers 230, panelist computers 210, and survey targ e t 
eomptrtefs-36 0 ar e each connect e d to the n e twork 170. Th es e users ac c es s the surv e ys using 
a n e twork - c lie nt a ppl i cat i on, preferably a web browser. 

In the pr e f e rred embodiment, the access to the survey bui l der and surv e ys is via th e 
World W i de Web (Web) u s ing web seiners and w e b browsers ( i .e. web c l ients); however, in 
alternate embodiments the server site and client site could be i mplemented as another typ e 
of n e twork serv e r and the c l ient software tt3 another typo of n e twork applicatien<rather than 
a web browser). 
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Respondent members 1 20 and panelists \ 40 aloo may oecuroly acce s s tho florv e r web 
si te 1 10 via the network 170, for e xamp l e* to s ign up for - mcmbership, vi e w - as s ignm e n t 
not i fications, or to manag e information about th e mselve s . 

Fig 3A through 3B An Illustrative Exampl e 

i n e xplaining th e operation and features of th e survey nyatom, it may bo helpful to 

c ons i der an illustrative example. Figs 3A, 3B, and 3C illustrat e thr ee v e rsion s of a s u r vey 

that- might be giv e n by-a hypothetical po l itical candidate at different pointo during that 

candidat e 's first four year t e rm in offic e . Th e candidate has - c e rtain objectives such as being 

popular - , - b eing considered as performing w el l, nndj m a ot important being re e lected. 

Survey I (301), shown in Fig 3A, ir . taken - 4uring the second year in office. Survey 2 
(302), s he wn in F i g 3B, i staken during the third - year in office. Survey 3 (303), shown in Fig 
3C is taken during the fourth yeor in office, on th e day before El e ction Day t 

Fig 3 A shows survey 1 (301) composed of various quoctiono design e d to measur e the 
c andidate's objectiv e s of popularity 310, performance 320, and re election 330. Survey 1 
(30 1 ) has different typos of qu e stions with diff e r e nt types of answ e rs. For exampl e , th e 
popularity and re el e ction qu e stions (311 and 331, respect i v e ly) have - ' -ye^- and "no" 
answers (se e 312 315 and 332), and the p e rformance question (32 1 ) i s a dir e ct respon se 
qu e stion having a lim i t e d rang e of answ e rs va l u e s (i. e . 1 5) (s ee 322 326). 

Fig 3B shows survey 2 (303) also composed of various questions designed to 
measure th e candidate's ■ objectives of popularity 310, performance 320, and re elect i on 330. 
However, now that th e cand i date has b e en i n offic e for ov e r two year s , the candidat e has 
becom e mor e-in tere s ted on how con s tituents stand on th e certain issues. Survey 2 (302) 
adds a new obj e ctive. - stand on issues 360. A new type of question is used. In question 361, 
answers are a form of multiple choice wh o re the choic e s i ndicat e a continuum of agreem e nt. 
Ther e w e r e a l f>o changes in th e qu e stions betwe e n surv e y 1 (301) and surv e y 2 (302). For 
example, affable 346 and articulate 347 wore added to measure th e popularity objectiv e 3 10. 
A l so, in the performanc e obj e ctive 320 questions, dom e st i c - pol i cy 321 wag dropped and 
re plac ed w i th foreign policy 351, social ooeurity 352 and defense 353. 

Fig 3C shows -survey 3 (303) aloo composed of various quest i ons dooign e d to 
measur e the cand i date's objoeiivea of popular i ty 310, performance 320, stand on issues 360, 
and r e election 330. — Howev e r, on tho e vo of r e ele ction, th e candkiate n e edq to know how 
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tbe-fe cent disaster and the war ore going to affect re e l ection, and i f the cand i date current l y 
ha s enough votes to win. Time is of the e s sences; baa e d on the immediate r e sults of th e 
survey, the -e andidate wi l l s pend tim e and money to ensu re that the objectiv e of re elec t ion i s 
m e t Surv e y 3 (303) uses a new typ e of qu es tio n to measure th e r e election objectiv e 330. 
The "wi l l vote v question - 378 i s in the form of a positive stat e ment about the vote and th e 
answers 379 are true or false. There wer e-riso-ot her - ohang e s in the questions between 
survey 1 (30 L Fig 3A), survey 2 (302, Fig 3B) and now survey 3 (303). For example, ' 
affable 3^16, which was add e d in surv e y 2 (302, Fig-3B ) rhn s been - replaced with -good 
speak e r" 386 in surv e y 3 (303, Fig 3C) to measure th e popularity objectiv e 3 10 i n a slightly 
differ e nt - 1 way.- Also; i n th e performance ob j ect i ve 320 questions* foreign po l icy 351 and 
soc i al security 352 wer e dropp e d and disast e r 354 and the war 359 w e r e add e & The 
questions regarding constituents* stand on issues were al l changed Tho socia l security 
question 363 was inv e rted by removing the word "NOT" and plac e d first as quest i on 365 
(modified social security question). Questions on citato tax 366 and going to war 367 
replaced the questions about social s e curity inv e stm e nt 362 and social programs 364, 

In this example, oaoh of the tliroe versions of the candidate's survey, i.e. Gurvoyo 1 
through - - 3 (301 through 303), had a diff e r e nt numb e r of response s , diff e rences in the 
object i ves be i ng measured; -and differences between sim i lar questions. As stated abov e , 
t he s e diff e r e nc e s - mak e it difficult to compar e the results of one version of the survey to 
another. The present invention provides systems and methods that include determin i ng an 
i nd e x - for each survey that is a measure of tho responses aga i n s t the goals and object i ves of 
the survey. The index for each survey i s a measure against a b e nchmark that transcends the 
chang e s inth e quest i ons;; questions typ e s, and objectiv e s of each ver s ion of - the survey. 

How Benchmarks Work? 

As part of the survey design, clients 130 can define benchmarks. Goals and weights 

are used to evaluate an index that quantifies the "success" of the survey. The index works in 

much the same way that a stock market index works (by selecting key stocks, weighting 

them and monitoring their activity as an indication of the overall stock market activityJ.-^Pbis 

will be explained, in tho following s e ction, by referring hack to the i l lustrat i ve e xampl e of 

th e hypothet i ca l po l itical candidate. 
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Fi t ; 4 Tallying R es ponses to Determine * an Ind e x 

F ; i g -4 i s a - tab ie- i >t ustFatiBg - 1:h e goals; we i ghts, and index calculat i on s for the 

questions i n the exampl e .survey 1 (301 ) of Fig 3A. In practice, th e surv e y i s respond e d to 

by a larg e number of respondents ■ ( 1 - 20) | however, - for - simplic i ty Fig <\ shows only th e 

responses of four hypothetical r e spond e nts (shown in. columns 4 30 a , 430b, 4300, and 430d, 

resp e ctively). The f i rst column refers to the questions from survey \ (301) in the some order 

as on F i g 3A. 

In ord e r to calculate nn index, th e r e spanofr - to each q u e stion first must be translat e d 
to a numb e r value: A numb e r of qu e stion types are known in the - art D i ff e rent translations 
ar e used for e ach type of question. 

Target values can be assigned to any number of questions on the survey. 

Weights are then applied to these questions indicating the impact any one question 
has on the overall survey index, relative to other questions on the survey. 

An index is then calculated for every survey that is completed by a respondent 
The index is a percentage that indicates the proximity of the survey answers to the desired 
goals with 100% indicating that the completed survey likely meets the desired goals. 

The surv ey - sy st em - hag a predetermin e d translation for each qu e stion typo. Th e 
fol l ow i ng table show s s ome exampl e translation s . 



RESPONSE 


VALUE 


Strongly disagree 




i Tend to disagree 


4 


Don't know/No opinion 




Tend to agree 




Strongly agree 






* 




a 


TRUE 


+ 


FALSE 


9 



For more complex quection typos (b.g. "ch e ck all that apply" or matrix responses), 
other types of calculations are made. Examples of functions usod in more compl e x 
calculat i ons are: 

jt — SUM added up all the values of the responses 
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* — COUNT count i ng the number of values greater tha n z ero 

* — EXISTS returning 1 for each item that is marked 

A — NOT EXISTS — returning 1 for each it e m that i s not marked 

* — ROW COUNT counting the number of rows in a matrix that has an an s w e r 

*~ — SUBSTRACT subtract the value from a pr e determined constant 

A — MULTIPLE multiple tho value by a predetermined constant 

A - — NOT convert 0 to 1 and vioa v e rsa. 

Th e surv e y d e signer ( e .g. a c l ient uo e r 130) celeota qu e st i on typeo from a 
predet e rmined se tof question typ e s supported by t e mplates in tho surv e y system - and - ent e r s 
th e text for each question and i ts possibl e responses. The surv e y system ahows th e surv e y 
designer the rang e of response values. For example, it) cr e ating survey 1 (301) the survey 
design e r first specifies the popularity objective 310. selects a matrix quest i on typo, e nt e rs th e 
word i ng for the question ( e .g. 3 - 1 1 ) and then - en te r s th e cub q ue stion wordi n g regard i n g 
honesty 312 with possibl e r e sponses of "y e a" and "no". In this cas e , th e purvey system 
would show a rang e of possible answer values of - either 0 (for "no") or I (for "y e o"). 
Likewise, anoth e r obj e cti v e and question typ e is e nt e red for th e performance obj e ctiv e 320. 
When e ntering tho sub question on tax es 322 the possible range of answer values 13 1 
through 5 with 5 being the best. Targ e t valu es can b e assign e d to any numb e r of qu e stion s 
on the aurv e y. The target value is the goal answer value (aoo ootumn A 10) for e aoh question. 
For e xampl e , for sub question on hon e sty 312, th e goal is I (m e aning an answ e r of y e s, and, 
for the sub question on taxea 322, the goal in only A (moaning good but not excel l ent). In 
this caso the goal i s not th e highest possible because our hypothet i cal cand i date has learn e d 
that you can't please a ll of tho people al l of the t i me. 

Weights are th e n appli e d to thes e qu e stions indicating the impact any on e qu e stion 
haft on - th e overal l survey index, r e lativ e to other questionson - the surv e y. Ih the e xamp le 
shown in Fig 4, the weight (see column 420) for th e sub question on honesty 312 is 8, the 
weight (column 420) for trustworthy 313 i s 4, and so forth. The candidate has set tho w e ight 
(ootumn - 420) for the re election question (row 33 1) to 50. The w ei ght fur each - question - is 
the p e rcentage of the i ndex (row 45) that the quest i on wil l contribut e . In this e xampl e , th e 
sum of tho weights in the popularity objective 312 315 i s 20 or 20% of th e ind e x. The sum 
of the weights i n the performance objective 320 is 30 or 30%. The ro election que s tion and 
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o bjective is 50% of the index for survey 1 (301). As shown at row 150 the sum ef4he 
weights i n co l umn 420 i s 1 00 . - - Fo r survey - 2 (302), the flame questions (e.g. 3 12, 313. and so 
forth) would have d i fferent weight v a lues because - there - are more quest i ons (e.g. 316 and 
307) and an add i tional obj e ctive (i.e. stand on issues 360). When modifying surv e y I (310) 
to form survey 2 (302), the survey design e r ass i gns new goal anowor values and weights; 
with th e gum of th e weights b ei ng 10Q. 

A^4ftdexH^s then calcu l ated for ev e ry s urvey that io oomplotod by a renpond e nt. 
Co l u m n 430a shows the responses by respond e nt I. Column 440n ahowo th e calculat e d 
va lue for each response. Th e calculated valu e is the response answer - value - divided by the 
goa t answ e r va l ue ■ mu l tipl ie d by th e weight 

Calo -- " (Response /Coal )*Woiglvfc 

The first respond e nt answered the hon e sty sub question 312, ycs» which translates to 
a r e spons e va l ue of 1 and calculated volue of 8. The tru s tworthy sub question 31 3 waa also 
answer e d yes, which translates to t - and ca l culat e s to 1 becaus e of th e low e r w ei ght The 
response volueo are translated and calculated for each quotation and on ind e x i o cc i loulat e d as 
a perc e ntag e against the b e nchmark of 100%. The index for the aurvey as o whole (or for 
any obj e ctiv e ) i o calculated by oumming the calculat e d va l ues and divid i ng by the sum of th e 
weights and multiplying by 100. 

Indox ~ (SUM( (Rooponco/Coal)*W e ighfc) / SUM<Woight) ) * 100 

It shoutd - bo understood that a numb e r cou l d bo ca l culat e d by applying goal anow e r 
values and weights to th e r e spons e s to all of the qu e gtions that measure any particu l ar 
objectiv e . Althoughnot shown in Hgs 1 through 6 for simplicity, a sub - ind e x for each 
objective could also be calculated and graph e d an a s e parat e graph element 

A differ e nt respondent index io calculated for each respon d ent to survey K Th e 
se cond respondent answered the honesty sub question 312, no, which tran s lates to a 
r e sponse value of 0 (soe column 130b) and calculated value of 0 (ooo co l umn 440b), The 
third r e spondent answ e r e d the honesty sub qu e stion 312, no, wh i ch translat e s to a respons e 
valu e of 0 (see co l umn 430c) and calculat e d value of 0 ( s ee - column 440c). (The thir d 
respond e nt does not particularly like th e candidate an d will not re e lert th e cand i date, see 
row 33 1 ). 
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As shown i n row ^150 of Fig 1, the index for respondent 1 015Qn) was 107.5, 
indicating that for this respondent the candidate has eftcoodod the goal. Th e indices of 
retipondento 2, 3, and 1 ( - 150b, 450c, 150d) were 82, H;5, and 97,5 respective l y. - - T - h e i nd i c e s 
of each r e spondent io added together and d i vided fay the number of respondent!) to calcu l at e 
an average index for a version of a surv e yv - whioh in thio example - for survey I is 75.1 (460a). 
The averag e of nil th e r e spondent indices for a version of a surv e y io the overall index for 
that v e rsion of the surv e y. 

If r e opondents answ e r each qu e otion with the corresponding goal - answ e r valu e , th e n 
the i nd e x for the survey will b e 100, indicating that all the objectiv e n to be measur e d by that 
survey have been m e t auoueoofully» G e nerally if th e r e opondents answer b e low the goat 
answer valu e s, th e ind e x wi l l be less - than 100 and if the respondents answ e r abov e th e goal 
answ e r values the i ndex will be gr e at e r than 100. However, b e cause of weighting, a high 
a n s w e r va l ue on a relat i v e ly higher - weighted question will have a more posit i ve impaot on 
di e ind e x. 

fig 5 Survoy Results 

Th e index i s a percentag e that i n di cates th e prox i mity of the ot i rvey (mowers to the 

d e sir e d goals with 100% i ndicating that the comp l eted survey li kely m e ets the d e sir ed goals. 
Fig 5 oftowo a graph 500 of the i ndividual r e spondent indices (150a, 150b, 450c, and 
150d) for survey f (30 1), The results lin e 530 of the survey 1 graph 500 show s that 
re s pondent 1 was abov e the survey benchmark 5 10, respondents 2 and A were a l so do n e to 
th e b e nchmark, but respond e nt 3 was far below th e benchmark Th e ourvey benchmark 5 1 0 
io an ind e x for a hypoth e tical re s pond e nt who answered the goal unw e r value for e very 
question on the survey, thus it is a num e rical way of r e presenting that c voty goal and 
objecti v e- Tor - the survey was m e et. A graph showing the individual respondent i ndices i s 
valuabl e to determ i ne the level of support the candidate has on an individual basic. 

Fig 6 Survey Trond - Craph 

The index can then be used to compare or rank surveys. 

■ As th e survey designer modifi e s a survey, design i ng new v e rsions, e .g. surv e y 2 
(302. Fig 3B) and survey 3 (303, Fig 3C),-the survey designer reevaluates the ob jectives and 
adjusts the quest i ons ao decked. N e w goal answer value s and weights ar e ass i gned to each 
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quest i on, guoh that the sum of the w e ights i s k e pt at 100, F ol lowing t ho stop s d es cribed in 
relation to Fig 4 for - each version, of tho survey, the survey system tran n la t o a o aoh re sp o ns e to 
on answer valu e , applies goal answ e r values and weights, and calculates an ind e x for each 
respondent and for all of the respondents of each v e rsion of the survey. 

Fig 6 shows a surv e y tr e nd graph/600 of the i ndices of each version of the survey 
(,t69ft, - 46 Q t), cwd 460c, reflpeottvet y^T fa e trend - l ine-620 of th e surv e y tr e nd graph 600 
s hows that the candidat e had come closer to meeting th e obj ec tives at the tim e of survey 2 
w i th a surv e y Z index of 92 (160b), but on the day bofore Election Day had dropped to - a 
surv e y 3 i ndex of 63 (160o), The benchmark l i ne 610 s hows th e su r vey benchmark (e.g, 
510) for each vers i on of th e survey. Lik e the benchmark us e d to m e asure water luvelu in th e 
ri ver e xamp l e, th e benchmark l in e 610 is a permanent value that can be used to compar e 
indice s from different v e r si ons of surv e ys, A graph showing the trend of survey i ndices 
ov e r tim e-i-fr- valuablo to determin e th e l e v e l support - the candidut e has over time. On th e day 
before th e elect i on, it is important to bo able to mok e chang e s to yet another version of the 
survey and get immediate results, especially i f the l e vel of support is decreasing in a trend 
that w i ll end up betow the percentage of votes needed to win the e l e ction. 

yice th e b e nchmark us e d to measur e and determine water levels on the river, the 
survey indices can be used to - m e asure and graph rcsponoos relative to obj e ctives ov s r - a 
vers i on of surveys. 

The Survey System 

In the preferred embodiment, the survey system is a turnkey market research 

application, available as a subscription-based service through secure Internet access. The 

service may be accessed from anywhere, anytime through a browser (as shown In tho 

pref e rred embodim e nts in Fig 1 mid F i g 2) or as a standalone application (not showu)^ As 

an Application Service Provider (ASP), the service is the only cost There is no need to 

purchase hardware, database and application software; they are all included. This service 

utilizes the system's proprietary technology to deliver a robust, cost-effective and intuitive 

method for conducting all types of traditional (paper and telephone) and web-based market 

research: 

1. Attitude and Usage 

2. Concept Testing 
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3. Advertising Testing 

4. Package/ Desi gn Testi ag 

5. Employee Satisfaction/Feedback 

6. Promotion Testing 

7. New Product Testing 

8. Customer Satisfaction 

9. Product Registration 

10. Respondent Screening 

The inv e ntion in not limited to the i ll u s trative e xample of tho politico! candidat e . 

The survey system addresses many of the issues plaguing the market research 
industry today. Research professionals will have the ability to focus on performing research 
without the current headaches and expenses associated with the acquisition and management 
of the technology and resources regularly required to perform research tasks in-house. The 
survey system is the complete, turnkey market research solution available from anywhere, 
anytime. 

Fig 1- Survey System Process Flow 

Fig 1 illustrates the process flow of the survey system. The survey system comprises 
a secure server 100, a server web site 1 10, a client web site 150, each connected to a 
network 170. 

The secure server 100 comprises a member database, a client survey database, 
survey templates database, a client respondent/other data database, and a respondent 
member database. 

Client users 130, namely client marketers and managers, signup for membership, 
design, post, and obtain reports, select respondent demographics, and maintain client 
respondent and other data via the server web site 1 10, 

Individuals who are interested in taking surveys, namely respondent members 120, 
sign up as respondent members 120 via the server web site 1 10 and when notified of a 
survey request, respond to the survey via the client web site 150. 

Individuals who match the desired demographics, namely, survey targets 160, are 
notified of a survey request and then respond to the survey via the client web site 150. 
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Professionals who have shown a particular understanding or discernment on a 
subject are invited to be part of a panel. These panelists 140 are notified of a survey request 
and then respond to the survey via the client web site 150. Panelists may be paid for their 
participation on the panel. 

The survey builder 180 provides a means for client users 130 to define, post a 
version of a survey, and obtain instant, concurrent results. The survey system allows for 
certain goals and weights to be associated with a survey. Each version of a survey is 
benchmarked based on the response's relationship to the goals. These responses contribute 
to an index or benchmark for a survey (for example, see Ftg 5) . The benchmark can be 
charted over time and over various versions of a survey or set of surveys to determine 
valuable trend information (for example, s oo- Rg - 6) . As shown in Fig l f the instant, 
concurrent results and reports are always available-24/7. 

Best Mode 

The best mode implementation for this invention is an Oracle database server with a 
database format as shown in Fig 1. This server 180 would implement a number of programs 
(collectively, the survey builder 180) that would allow researchers to create surveys 
comprising a number of different survey questions of various types. The researcher (e.g. 
client user 130) would assign goals for a survey and assign weights to the answers to each 
questio n ( e .g. F i g 1, oohunn 420 nnd 430) . The results for multiple respondents would be 
tallied and the weights applied to determine an index (e.g. 460a of Fig 1 and F i g 5). Further 
versions of a survey or surveys with similar goals could be correlated by the index-(«hrgr 
460a, 460b, 106c of Fig 6) . The programs display the indices of various surveys in the form 
of a graph (eg. trend graph 600 of Fig 6) . All survey creation tools, the surveys themselves, 
and the reports including the indices would be available from the system via the World Wide 
Web or via standalone client applications. 

The Day Before the Election 

As discuss e d above, using - the survey f i ystem researchers can define and ref i ne 

v e rsions of n survey, post th e surv e ys to their w e b oit e a, notify r e upondonto. and obta in 
instant, concurrent survey re s ults. - Return i ng to our hypothetical pol i t i cal candidate, on the 
doy b e fore ths election, the candidate can post any number of - versions of a survey to obta i n 
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fft ston i reaction to events of the day, such as campaign ads, qpoechcG> or pu b l ic app o nranc o s 
i n - contested t i roao. Questions can bo refined to zero i n on particular isnuas or evcnta. - 
Survoy targotn can be modif i ed to target particular groups or geographic aroa o . Although, tho 
surv e ys awd - surv e y question or e diff e r e nt, the candid a te can see tha positiv e and negat i v e 
effect s of various actiona or events by rec e iving the ind e x trend graphs and - c rt h e PToportfi. 
Hav i ng th e ability to r e ceiv e immed i ate response provi'd a fta Yahioblo and timely dec i s i on 
guiding resourc e . 

Conclusion, Ramification, and Scope 

Accordingly, the reader will see that the survey system with the built in 
benchmarking and index graphs of the present invention provides a means of creating 
surveys, assigning goals and weights, tallying results to determine an index, and graphing 
related indices to show trends. 

Furthermore, the present invention has additional advantages in that; 

(a) it provides a robust survey creation tool; 

(b) it provides for almost instant results of surveys, especially web-based surveys; 

(c) it provides a way for researchers to easily modify an existing survey and then 
correlate data from it with that collected via an earlier survey; 

Although the descriptions above contain many specifics, these should not be 
construed as limiting the scope of the invention but as merely providing illustrations of some 
of the preferred embodiments of this invention. For example, the format of the index graphs 
can have different styles, and the same relative operation, relative performance, and relative 
perceived value would result Also, these processes can each be implemented as a hardware 
apparatus that will improve the performance significantly. ■ Further, the; e xample surveys of 
the i llustrativ e e xampl e procont e d queationa ordorod by th e objective (e - jg. popular i ty 
fol l ow e d by performanc e and re elect i on), i n practic e tho ourvoy system pr e ferably present a 
qu e stions in any order, including randomly to each roopondon fc- 

Thus the scope of the invention should be determined by the appended claims and 
their legal equivalents, and not solely by the examples given. 
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DATA-DRIV EN INTERNET SURVEY SYSTEM 
BENCHMARKING SURVEYS 
Abstract: Methods and machines that provide for creation of surveys with a variety 
of questions. Each survey is assigned goals. Each question is weighted toward the goals. 
The results of each response to a survey i sarg tallied to determine an index. The index is 
correlated with the indices of other related surveys to produce a graph of the trends of the 
success toward the goals. 
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